Neural network learning device, neural network learning method, and recording medium on which neural network learning program is stored

ABSTRACT

A neural network learning device 20 is equipped with: a determination module 22 that determines the size of a local region in learning information 200 which is to be learned by a neural network 21 containing multiple layers, said determination being made for each layer, on the basis of the structure of the neural network 21; and a control module 25 that, on the basis of size of the local region as determined by the determination module 22, extracts the local region from the learning information 200, and performs control such that the learning of the learning information represented by the extracted local region by the neural network 200 is carried out repeatedly while changing the size of the extracted local region, and thus, a reduction in the generalization performance of the neural network can be avoided even when there is little learning data.

This application is a National Stage Entry of PCT/JP2018/001151 filed onJan. 17, 2018, which claims priority from Japanese Patent Application2017-007614 filed on Jan. 19, 2017, the contents of all of which areincorporated herein by reference, in their entirety.

TECHNICAL FIELD

The invention of the present application relates to a technique forcontrolling learning by a neural network.

BACKGROUND ART

In recent years, because of rapid development of a technique related toartificial intelligence (AI), a neural network constituting AI has beendrawing attention in various technical fields. As one such technicalfield, there is a pattern recognition technique. A pattern recognitiontechnique is a technique for estimating which class (category) an inputpattern belongs to. As a specific example related to such patternrecognition, object recognition for estimating what an object includedin an input image is, voice recognition for estimating a speech contentfrom input voice, or the like is cited, for example.

In a pattern recognition technique, statistic machine learning is widelyutilized. Further, in recent years, because of rapid development of alearning technique called deep learning, a neural network has beenbecoming able to perform accurate recognition of a diversely fluctuatinginput pattern without being affected by the fluctuation. In view of sucha background, there is a rising expectation for a technique forefficiently performing learning by a neural network, and enabling aneural network to more accurately make a determination.

As a technique associated with such a technique, PTL 1 discloses animage processing device which shortens a learning time of a multi-layerneural network when a registered category is updated. This multi-layerneural network performs processing of outputting an output value to aprocessing unit in a layer at a next stage, on the basis of a result ofapplying spatial filters differing from each other to an input image.This multi-layer neural network includes an anterior layer and aposterior layer. The anterior layer finally generates each of aplurality of feature amounts. The posterior layer finally classifies aninput image into one of a plurality of categories by performingprocessing of outputting an output value to a processing unit in a layerat a next stage on the basis of a result of applying weighting factorsdiffering from each other to the plurality of feature amounts. When acategory is updated, the image processing device corrects a value of aweighting factor in a posterior layer without correcting a value of afilter in an anterior layer.

Furthermore, PTL 2 discloses a learning device which performs learningby a forward propagation type multi-layer neural network throughsupervised learning. This learning device performs learning by costcalculation using a training data set with regard to a multi-layerneural network having, in a hidden layer, a probing neuron which doesnot forwardly propagate to an output layer. Then, after removing anupper layer of the multi-layer neural network on the basis of a cost ofthe probing neuron, this learning device designates a probing neuron ofa remaining uppermost layer as an output layer.

CITATION LIST Patent Literature

[PTL 1] Japanese Unexamined Patent Application Publication No.2016-139176

[PTL 2] Japanese Unexamined Patent Application Publication No.2015-095215

SUMMARY OF INVENTION Technical Problem

In deep learning performed by a neural network, there are a considerablenumber of parameters to learn, and thus, a large amount of learning datais generally needed. Moreover, it is known that, when an amount oflearning data is small, generalization performance is significantlyreduced due to overfitting. Overfitting refers to a situation where aneural network can be adapted to input learning data, but cannot beadapted (i.e., cannot be generalized) to unknown data.

In deep learning, learning is generally performed on the basis of backpropagation. In back propagation, by backwardly propagating an errorfrom an output layer toward an input layer in a multi-layer neuralnetwork, learning is performed in such a way that the error in theoutput layer is decreased. When an amount of learning data is small, aneural network can sufficiently decrease an error by learning only in ahigh-order layer close to an output layer. In this case, there occurs aphenomenon in which learning in a high-order layer advances, andlearning in a low-order layer close to an input layer does not advance.Thus, a neural network gets into an overfitted state, and there is aproblem that generalization performance of the neural network isreduced. PTLs 1 and 2 do not refer to this problem. A main object of theinvention of the present application is to provide a neural networklearning device and the like which solve such a program.

Solution to Problem

A neural network learning device according to one aspect of theinvention of the present application includes: a determination means fordetermining a size of a local region in learning information which istargeted for learning by a neural network including a plurality ofhierarchical layers, for each of the hierarchical layers, based on astructure of the neural network; and a control means for controlling insuch a way as to repeat extracting the local region from the learninginformation, based on the size of the local region which is determinedby the determination means, and learning, by the neural network, thelearning information being represented by the local region beingrepeatedly extracted, while changing the size of the local region beingextracted from the learning information.

In another aspect of achieving the above-described object, a neuralnetwork learning method according to one aspect of the invention of thepresent application includes: by an information processing device,determining a size of a local region in learning information which istargeted for learning by a neural network including a plurality ofhierarchical layers, for each of the hierarchical layers, based on astructure of the neural network; and controlling in such a way as torepeat extracting the local region from the learning information, basedon the size of the local region which is determined, and learning, bythe neural network, the learning information being represented by thelocal region being extracted, while changing the size of the localregion being extracted from the learning information.

Moreover, in a further aspect of achieving the above-described object, aneural network learning program according to one aspect of the inventionof the present application which causes a computer to execute:determination processing of determining a size of a local region inlearning information which is targeted for learning by a neural networkincluding a plurality of hierarchical layers, for each of thehierarchical layers, based on a structure of the neural network; andcontrol processing of controlling in such a way as to repeat extractingthe local region from the learning information, based on the size of thelocal region which is determined by the determination processing, andlearning, by the neural network, the learning information beingrepresented by the local region being extracted, while changing the sizeof the local region being extracted from the learning information.

Furthermore, the invention of the present application can also beimplemented by a computer-readable and non-volatile recording medium onwhich the neural network learning program (computer program) is stored.

Advantageous Effects of Invention

The invention of the present application is able to avoid reduction ofgeneralization performance of a neural network even when learning dataare small in amount.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a neuralnetwork learning device 10 according to a first example embodiment ofthe invention of the present application.

FIG. 2 is a diagram exemplifying a hierarchical structure of a neuralnetwork 11 according to the first example embodiment of the invention ofthe present application.

FIG. 3 is a diagram exemplifying a relation between a convolutionalcalculation performed by an intermediate layer included in the neuralnetwork 11 according to the first example embodiment of the invention ofthe present application, and a size of a local region targeted forlearning.

FIG. 4 is a diagram exemplifying a relation between a size of a localregion determined as a learning target in first learning processing by adetermination module 12 according to the first example embodiment of theinvention of the present application, and a minimum value of a size of alocal region at which a learning effect is expected in each layer.

FIG. 5 is a diagram exemplifying a relation between a size of a localregion determined as a learning target in second learning processing bythe determination module 12 according to the first example embodiment ofthe invention of the present application, and a minimum value of a sizeof a local region at which a learning effect is expected in each layer.

FIG. 6 is a flowchart illustrating an operation of the neural networklearning device 10 according to the first example embodiment of theinvention of the present application.

FIG. 7 is a block diagram illustrating a configuration of a neuralnetwork learning device 20 according to a second example embodiment ofthe invention of the present application.

FIG. 8 is a block diagram illustrating a configuration of an informationprocessing device 900 which can execute a neural network learning deviceaccording to each example embodiment of the invention of the presentapplication.

EXAMPLE EMBODIMENT

Hereinafter, example embodiments of the invention of the presentapplication are described in detail with reference to the drawings.

First Example Embodiment

FIG. 1 is a block diagram schematically illustrating a configuration ofa neural network learning device 10 according to a first exampleembodiment of the invention of the present application. The neuralnetwork learning device 10 is a device which performs deep learning byinput learning data 100 (learning information), and performs processingsuch as pattern recognition, based on a result of the learning. Thelearning data 100 are, for example, supervised data including acombination of data (problem) representing a target for which patternrecognition is performed, and a recognition result (answer) to beexpected.

The neural network learning device 10 includes a neural network 11, adetermination module 12, and a control module 15.

The neural network 11 is a multi-layer neural network including aplurality of hierarchical layers (layers), and performs deep learning byusing input learning data 100. When input data (pattern recognitioninput data 101) representing a target for which pattern recognition isto be performed are input, the neural network 11 performs patternrecognition processing on the basis of a learning result, and outputs aresult of the processing as a pattern recognition result 102. As thispattern recognition processing, the neural network 11 performs, forexample, face recognition processing, object recognition processing,voice recognition processing, or the like.

FIG. 2 is a diagram conceptually exemplifying a hierarchical structureof the neural network 11 according to the present example embodiment.The neural network 11 includes an input layer 110, an L1 layer 111, anL2 layer 112, an L3 layer 113, and an output layer 114. The L1 layer111, the L2 layer 112, and the L3 layer 113 are intermediate layers(hidden layers). An intermediate layer according to the present exampleembodiment has a three-layer configuration, but an intermediate layer isnot limited to a three-layer configuration.

The input layer 110, the L1 layer 111, the L2 layer 112, the L3 layer113, and the output layer 114 are, for example, units each having aneuron element (not illustrated) imitating a neural circuit of acreature. In the present example embodiment, the output layer 114 sideis defined as a high-order side (next-stage side or posterior-stageside), and the input layer 110 side is defined as a low-order side (ananterior-stage side).

When the neural network 11 performs pattern recognition, theintermediate layers (i.e., the L1 layer 111, the L2 layer 112, and theL3 layer 113) each perform recognition processing for data representinga recognition target after data (image data, voice data, or the like)representing the recognition target are input to the input layer 110,and the output layer outputs a result of the recognition.

The neural network 11 according to the present example embodiment isconfigured as, for example, a convolutional neural network (CNN) whichis well known in a technical field such as deep learning. In a“convolutional” calculation performed in a CNN, for example, when imagedata representing a recognition target are represented as a matrix (eachelement of a matrix is equivalent to one pixel), a sliding windowfunction (filter) and the matrix are multiplied element by element. Inthis “convolutional” calculation, overall convolution is then performedby performing, for each element, a calculation of summingelement-by-element multiplication values while sliding a filter (kernel)in such a way as to cover the entire matrix. It is assumed that all ofthe L1 layer 111, the L2 layer 112, and the L3 layer 113 according tothe present example embodiment perform a convolutional calculation by afilter (hereinafter, referred to as a “3×3 filter” in the presentapplication) having a size of “3×3”. However, a size of a window when,for example, a convolutional calculation is applied to theseintermediate layers by use of a filter is not limited to “3×3”.

Recognition processing performed by intermediate layers included in theneural network 11 becomes higher in level from a layer located on alow-order side toward a layer located on a high-order side. For example,when the neural network 11 performs face recognition processing, the L1layer 111 performs processing of detecting an edge from face image data(pixel data) representing a recognition target. The L2 layer 112performs processing of detecting a simple shape (an element or the likeconstituting a face) in a face image by using the edge detected by theL1 layer 111. Then, the L3 layer 113 performs processing of detecting amore complicated feature needed for identification of a person, by usingthe simple shape detected by the L2 layer 112.

FIG. 3 is a diagram schematically exemplifying a relation between aconvolutional calculation performed by an intermediate layer included inthe neural network 11 according to the present example embodiment, and asize of a partial region (hereinafter, referred to as a “local region”in the present application) in the learning data 100 input to the inputlayer 110 which is targeted for learning by the neural network 11.

The L3 layer 113 performs a convolutional calculation by a 3×3 filterfor an output from the L2 layer 112. In this case, a size of a region(hereinafter, referred to as a “learning target region”) targeted forlearning in the L2 layer 112 related to (based on) one element (regionsize=1×1) in the L3 layer 113 becomes “3×3”. The L2 layer 112 performs aconvolutional calculation by a 3×3 filter for an output from the L1layer 111. Therefore, a region of “5×5” being a region in which a regionof “3×3” in the L2 layer is movable by a maximum of two elements in arow direction and a column direction is a learning target region in theL1 layer 111 related to the element in the L3 layer 113. The L1 layer111 performs a convolutional calculation by a 3×3 filter for an outputfrom the input layer 110. Therefore, a region of “7×7” being a region inwhich a region of “5×5” in the L1 layer is movable by a maximum of twoelements in a row direction and a column direction is a learning targetregion in the input layer 110 related to the element in the L3 layer113.

FIG. 3 represents the above-described relation, and illustrates that aminimum value of a size of a learning target region in the input layer110 at which a learning effect is expected in relation to the L3 layer113 is “7×7”. In other words, when a size of a learning target region isless than “7×7”, valid data exist only in a central part of a filter inrelation to the L3 layer 113 and therefore, learning becomes difficult.Similarly, a minimum value of a size of a learning target region in theinput layer 110 at which a learning effect is expected in relation tothe L2 layer 112 is “5×5”. Moreover, similarly, a minimum value of asize of a learning target region in the input layer 110 at which alearning effect is expected in relation to the L1 layer 111 is “3×3”.

As described above, in the neural network 11, a layer located closer toa high-order side performs higher-level recognition processing ingeneral. Then, as recognition processing to be performed becomes higherin level, learning using a wider-range local region (learning targetregion) is needed. Therefore, in order to enjoy a processing resulthaving high accuracy during operation after learning, a minimum value ofa size of a learning target region needs to be small in relation to theL1 layer 111, and large in relation to the L3 layer 113.

The neural network learning device 10 according to the present exampleembodiment controls in such a way that the neural network 11 performs aplurality of times of learning in which a size of a learning targetregion is changed for the learning data 100 in consideration of a sizedifference of needed learning target regions related to theabove-described different intermediate layers. In other words, thedetermination module 12 illustrated in FIG. 1 determines, for each ofthe L1 layer 111, the L2 layer 112, and the L3 layer 113, a size of alocal region (learning target region) in the input layer 110 which istargeted for learning by the neural network 11, on the basis of astructure of the neural network 11. As a structure of the neural network11, the determination module 12 uses, for example, a size of a filterrelated to each of the L1 layer 111, the L2 layer 112, and the L3 layer113 which perform a convolutional calculation by the filter.

FIG. 4 is a diagram exemplifying a relation between a size of a learningtarget region determined as a learning target in first learningprocessing by the determination module 12 according to the presentexample embodiment, and a minimum value of a size of a local region atwhich a learning effect is expected in each layer. The determinationmodule 12 determines a size of a learning target region in such a way asto satisfy, in first learning processing, a condition of being equal toor more than a minimum value of a size of a local region at which alearning effect related to the L1 layer 111 is expected, and being lessthan a minimum value of a size of a local region at which a learningeffect related to the L2 layer 112 is expected. More specifically, asillustrated in FIG. 4 , a minimum value of a size of a local region atwhich a learning effect in the L1 layer 111 is expected is “3×3”, and aminimum value of a size of a local region at which a learning effect inthe L2 layer 112 is expected is “5×5”. Therefore, in this case, in orderto satisfy the condition, the determination module 12 determines a sizeof a local region determined as a first learning target, to be, forexample, “4×4”.

The determination module 12 then determines a size of a learning targetregion in second learning processing, on the basis of a structure of theneural network 11. FIG. 5 is a diagram exemplifying a relation between asize of a local region determined as a second learning target by thedetermination module 12 according to the present example embodiment, anda minimum value of a size of a local region at which a learning effectis expected in each layer. The determination module 12 determines a sizeof a learning target region in such a way as to satisfy, in the secondlearning processing, a condition of being equal to or more than aminimum value of a size of a local region at which a learning effect inthe L2 layer 112 is expected, and being less than a minimum value of asize of a local region at which a learning effect in the L3 layer 113 isexpected. More specifically, as illustrated in FIG. 5 , a minimum valueof a size of a local region at which a learning effect in the L2 layer112 is expected is “5×5”, and a minimum value of a size of a localregion at which a learning effect in the L3 layer 113 is expected is“7×7”. Therefore, in this case, in order to satisfy the condition, thedetermination module 12 determines a size of a local region determinedas a second learning target, to be, for example, “6×6”.

The determination module 12 then determines a size of a learning targetregion in such a way as to satisfy, in relation to third learningprocessing, a condition of being equal to or more than a minimum valueof a size of a local region at which a learning effect in the L3 layer113 is expected. Note that, in the present example embodiment, the L3layer 113 is a layer located on a highest order in an intermediatelayer, and therefore, the condition does not include “being less than aminimum value of a size of a local region at which a learning effect ina layer located at a next stage is expected”. Thus, in this case, thedetermination module 12 determines a size of a third learning targetregion to be, for example, “8×8”.

The control module 15 illustrated in FIG. 1 includes an extractionmodule 13 and a learning control module 14. On the basis of a size ofthe learning target region determined by the determination module 12,the extraction module 13 repeatedly extracts a local region from thelearning data 100 while changing a size of the local region. In otherwords, the extraction module 13 extracts a local region having a size of“4×4” from the learning data 100 in first learning processing. Theextraction module 13 extracts a local region having a size of “6×6” fromthe learning data 100 in second learning processing. The extractionmodule 13 extracts a local region having a size of “8×8” from thelearning data 100 in third learning processing. In this instance, theextraction module 13 may divide the learning data 100 into a size of alocal region determined by the determination module 12, or may permitoverlapping of a plurality of local regions and then randomly extract aplurality of local regions.

The learning control module 14 controls the neural network 11 in such away that the neural network 11 repeatedly learns with an input of localregions repeatedly extracted by the extraction module 13. In otherwords, the learning control module 14 controls the neural network 11 insuch a way that the neural network 11 learns with an input of a localregion having a size of “4×4” in first learning processing. In thiscase, as described above, the neural network 11 performs learningrelated to the L1 layer 111 because learning related to the L2 layer 112and the L3 layer 113 is difficult.

The learning control module 14 controls the neural network 11 in such away that the neural network 11 learns with an input of a local regionhaving a size of “6×6” in second learning processing. In this case, theneural network 11 performs learning related to the L1 layer 111 and theL2 layer 112 because learning related to the L3 layer 113 is difficult.Then, the learning control module 14 controls the neural network 11 insuch a way that the neural network 11 learns with an input of a localregion having a size of “8×8” in third learning processing. In thiscase, the neural network 11 performs learning related to the L1 layer111 to the L3 layer 113.

The neural network 11 performs the above-described learning by use of,for example, stochastic gradient descent or the like based on backpropagation, which is a learning method frequently used in general.

Next, an operation (processing) when the neural network learning device10 according to the present example embodiment is equipped with each ofthe above-described modules is described in detail with reference to aflowchart in FIG. 6 .

The neural network learning device 10 repeatedly executes processingfrom a step S102 to a step S104 while changing a variable i (i is one ofintegers 1 to 3) from 1 to 3 (step S101).

On the basis of a structure of the neural network 11, the determinationmodule 12 determines a size of a learning target region in such a way asto satisfy a condition of being equal to or more than a minimum value ofa size of a local region at which a learning effect in an Li layer isexpected, and being less than a minimum value of a size of a localregion at which a learning effect in an L(i+1) layer is expected (stepS102).

The extraction module 13 included in the control module 15 extracts alearning target region from the learning data 100 on the basis of a sizeof the learning target region determined by the determination module 12(step S103). The learning control module 14 included in the controlmodule 15 controls the neural network 11 in such a way that the neuralnetwork 11 learns with an input of a learning target region extracted bythe determination module 12 (step S104).

When a variable i is less than 3, the neural network learning device 10adds 1 to the variable i, and then executes processing from the stepS102, whereas, when a variable i is 3, the neural network learningdevice 10 ends the whole processing (step S105).

Note that, although the neural network learning device 10 repeatedlyperforms the steps S102 to S104 in the flowchart illustrated in FIG. 6 ,a flow performed by the neural network learning device 10 may bedifferent from this. For example, the neural network learning device 10may repeatedly perform S103 to S104 illustrated in FIG. 6 afterdetermining all sizes of learning target regions for the respectiveintermediate layers (i.e., after completing the step S102 for all valuesthat can be taken by a variable i).

The neural network learning device 10 according to the present exampleembodiment is able to avoid reduction of generalization performance of aneural network even when learning data are small in amount. A reason forthis is that the neural network learning device 10 determines a size ofa learning target region for each hierarchical layer on the basis of astructure of a neural network including a plurality of hierarchicallayers, and controls in such a way that the neural network repeatedlylearns with an input of a local region extracted in the size.

An advantageous effect implemented by the neural network learning device10 according to the present example embodiment is described below indetail.

It is known that, in deep learning performed by a neural network,generalization performance is significantly reduced due to overfittingwhen an amount of learning data is small. In deep learning, generally,learning is performed on the basis of back propagation in such a waythat an error in the output layer is decreased by backwardly propagatingan error from an output layer toward an input layer. When an amount oflearning data is small, a neural network can sufficiently decrease anerror by learning only in a high-order layer close to an output layer.However, in this case, there occurs a phenomenon in which learning in ahigh-order layer advances, and learning in a low-order layer close to aninput layer does not advance. Thus, a neural network gets into anoverfitted state, and there is a problem that generalization performanceof the neural network is reduced.

For such a problem, the neural network learning device 10 according tothe present example embodiment includes the determination module 12 andthe control module 15. Further, the control module 15 includes theextraction module 13 and the learning control module 14. In other words,the determination module 12 determines, for each hierarchical layer, asize of a local region in the input layer 110 which is targeted forlearning by the neural network 11 including a plurality of hierarchicallayers, on the basis of a structure of the neural network 11. On thebasis of a size of the local region (learning target region) determinedby the determination module 12, the extraction module 13 repeatedlyextracts a local region while changing a size of the local region. Then,the learning control module 14 controls the neural network 11 in such away that the neural network 11 repeatedly learns with an input of localregions repeatedly extracted by the extraction module 13.

More specifically, the neural network learning device 10 according tothe present example embodiment first controls in such a way that theneural network 11 learns with an input of a local region (learningtarget region) having such a size that learning related to the L1 layer111 advances, and learning related to the L2 layer 112 and the L3 layer113 does not advance. The neural network learning device 10 thencontrols in such a way that the neural network 11 learns with an inputof a local region having such a size that learning related to the L1layer 111 and the L2 layer 112 advances, and learning related to the L3layer 113 does not advance. Thereafter, the neural network learningdevice 10 controls the neural network 11 in such a way that the neuralnetwork 11 learns with an input of a local region having such a sizethat learning related to the L1 layer 111 to the L3 layer 113 advances.Thus, the neural network learning device 10 according to the presentexample embodiment is able to avoid reduction of generalizationperformance of the neural network 11 by controlling in such a way thatthe neural network 11 efficiently learns even when learning data aresmall in amount.

Furthermore, the determination module 12 according to the presentexample embodiment determines a size of a local region for a firsthierarchical layer in such a way as to satisfy a condition of beingequal to or more than a minimum value of a size of a local region atwhich a learning effect related to the first hierarchical layer isexpected, and being less than a minimum value of a size of a localregion at which a learning effect related to a second hierarchical layeradjacent to the first hierarchical layer is expected. Then, thedetermination module 12 determines a size of a local region on the basisof a size of a filter related to a hierarchical layer for which aconvolutional calculation by the filter is performed. Thus, the neuralnetwork learning device 10 according to the present example embodimentis able to certainly avoid reduction of generalization performance ofthe neural network 11.

Second Example Embodiment

FIG. 7 is a block diagram schematically illustrating a configuration ofa neural network learning device 20 according to a second exampleembodiment of the invention of the present application.

The neural network learning device 20 according to the present exampleembodiment includes a determination module 22 and a control module 25.

The determination module 22 determines, for each hierarchical layer, asize of a local region in learning information 200 which is targeted forlearning by a neural network 21 including a plurality of hierarchicallayers, on the basis of a structure of the neural network 21.

The control module 25 extracts a local region from the learninginformation 200 on the basis of a size of the local region determined bythe determination module 22.

The control module 25 controls in such a way that the neural network 21repeatedly performs learning of learning information represented by theextracted local region while changing a size of the local regionextracted from the learning information 200.

The neural network learning device 20 according to the present exampleembodiment is able to avoid reduction of generalization performance of aneural network even when learning data are small in amount. A reason forthis is that the neural network learning device 20 determines, for eachhierarchical layer, a size of a local region targeted for learning onthe basis of a structure of a neural network including a plurality ofhierarchical layers, and controls in such a way that the neural networkrepeatedly learns with an input of the local region extracted in thesize.

<Hardware Configuration Example>

In each of the example embodiments described above, each unit in theneural network learning devices 10 and 20 illustrated in FIGS. 1 and 7can be implemented by dedicated hardware (HW) (electronic circuitry).Moreover, in FIGS. 1 and 7 , at least the following configuration can beconsidered as a functional (processing) unit (software module) of asoftware program.

-   -   the determination modules 12 and 22, and    -   the control modules 15 and 25.

However, segmentation of each unit illustrated in these drawings is aconfiguration for convenience of description, and various configurationscan be assumed at implementation. One example of a hardware environmentin this case is described with reference to FIG. 8 .

FIG. 8 is a diagram exemplarily describing a configuration of aninformation processing device 900 (a computer) which can execute theneural network learning device according to each example embodiment ofthe invention of the present application. In other words, FIG. 8represents a configuration of a computer (an information processingdevice) which can implement the neural network learning devicesillustrated in FIGS. 1 and 7 or a part of the device, and a hardwareenvironment which can implement each function in the example embodimentsdescribed above. The information processing device 900 illustrated inFIG. 8 includes the followings as components.

-   -   a central processing unit (CPU) 901,    -   a read only memory (ROM) 902,    -   a random access memory (RAM) 903,    -   a hard disc (storage device) 904,    -   a communication interface 905 with an external device such as a        wireless transmission/reception unit,    -   a bus 906 (communication wire),    -   a reader/writer 908 which can read and write data stored in a        recording medium 907 such as a compact disc read only memory        (CD-ROM), and    -   an input/output 909.

In other words, the information processing device 900 including theabove-described components is a general computer in which thesecomponents are connected via the bus 906. The information processingdevice 900 includes a plurality of CPUs 901 in one case, or includes amulticore CPU 901 in another case.

Furthermore, the invention of the present application described with theabove-described example embodiments as examples supplies the informationprocessing device 900 illustrated in FIG. 8 with a computer programwhich can implement the following function. The function is a functionof the above-described configuration in the block configuration diagrams(FIGS. 1 and 7 ) referred to in the description of the exampleembodiments, or the flowchart (FIG. 6 ). Thereafter, the invention ofthe present application is accomplished by reading the computer programin the CPU 901 of the hardware, and then interpreting and executing thecomputer program. Further, the computer program supplied into the devicehas only to be stored in a readable/writable volatile memory (the RAM903), or a non-volatile storage device such as the ROM 902 or the harddisc 904.

Still further, in the above-described case, a general procedure can beadopted at present as a method of supplying a computer program into thehardware. As the procedure, there is, for example, a method ofinstalling into the device via various recording media 907 such as aCD-ROM, a method of downloading from outside via a communication linesuch as the Internet, or the like. Moreover, in such a case, it can beconsidered that the invention of the present application is configuredby a code constituting the computer program, or a recording medium 907storing the code.

While the invention has been particularly shown and described withreference to example embodiments thereof, the invention is not limitedto these embodiments. It will be understood by those of ordinary skillin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the present invention asdefined by the claims.

This application is based upon and claims the benefit of priority fromJapanese patent application No. 2017-007614, filed on Jan. 19, 2017, thedisclosure of which is incorporated herein in its entirety by reference.

REFERENCE SIGNS LIST

-   10 Neural network learning device-   11 Neural network-   110 Input layer-   111 L1 layer-   112 L2 layer-   113 L3 layer-   114 Output layer-   12 Determination module-   13 Extraction module-   14 Learning control module-   15 Control module-   100 Learning data-   101 Pattern recognition input data-   102 Pattern recognition result-   20 Neural network learning device-   21 Neural network-   22 Determination module-   25 Control module-   200 Learning information-   900 Information processing device-   901 CPU-   902 ROM-   903 RAM-   904 Hard disc (storage device)-   905 Communication interface-   906 Bus-   907 Recording medium-   908 Reader/writer-   909 Input/output interface

What is claimed is:
 1. A neural network learning device comprising: atleast one memory storing a computer program; and at least one processorconfigured to execute the computer program to determine a size of alocal region in learning information for each of a plurality ofhierarchical layers, based on a structure of a neural network, the localregion being targeted for learning by the neural network including thehierarchical layers; control in such a way as to repeat extracting thelocal region from the learning information, based on the size of thelocal region which is determined, and learning, by the neural network,the learning information being represented by the local region beingextracted, while changing the size of the local region being extractedfrom the learning information; and determine the size of the localregion for a first hierarchical layer of the hierarchical layers in sucha way as to satisfy being equal to or more than a minimum value of thesize of the local region at which a learning effect related to the firsthierarchical layer is expected, and being less than a minimum value ofthe size of the local region at which a learning effect in a secondhierarchical layer of the hierarchical layers being located at a stagenext to the first hierarchical layer is expected.
 2. The neural networklearning device according to claim 1, wherein the processor isconfigured to execute the computer program to determine the size of thelocal region, based on a size of a filter related to the hierarchicallayer which performs a convolutional calculation by the filter.
 3. Theneural network learning device according to claim 1, wherein the neuralnetwork which performs pattern recognition processing.
 4. The neuralnetwork learning device according to claim 3, wherein, as the patternrecognition processing, the neural network performs face recognitionprocessing, object recognition processing, or voice recognitionprocessing.
 5. A neural network learning method comprising: by aninformation processing device which has at least one memory storing acomputer program and at least one processor configured to execute thecomputer program, determining a size of a local region in learninginformation for each of a plurality of hierarchical layers, based on astructure of a neural network, the local region being targeted forlearning by the neural network including the hierarchical layers;controlling in such a way as to repeat extracting the local region fromthe learning information, based on the size of the local region which isdetermined, and learning, by the neural network, the learninginformation being represented by the local region being extracted, whilechanging the size of the local region being extracted from the learninginformation, and determining the size of the local region for a firsthierarchical layer of the hierarchical layers in such a way as tosatisfy being equal to or more than a minimum value of the size of thelocal region at which a learning effect related to the firsthierarchical layer is expected, and being less than a minimum value ofthe size of the local region at which a learning effect in a secondhierarchical layer of the hierarchical layers being located at a stagenext to the first hierarchical layer is expected.
 6. The neural networklearning method according to claim 5, further comprising: determiningthe size of the local region, based on a size of a filter related to thehierarchical layer which performs a convolutional calculation by thefilter.
 7. A non-transitory computer-readable recording medium storing aneural network learning program which causes a computer to execute:determination processing of determining a size of a local region inlearning information for each of a plurality of hierarchical layers,based on a structure of a neural network, the local region beingtargeted for learning by the neural network including the hierarchicallayers; and control processing of controlling in such a way as to repeatextracting the local region from the learning information, based on thesize of the local region which is determined by the determinationprocessing, and learning, by the neural network, the learninginformation being represented by the local region being extracted, whilechanging the size of the local region being extracted from the learninginformation; wherein the determination processing determines the size ofthe local region for a first hierarchical layer of the hierarchicallayers in such a way as to satisfy being equal to or more than a minimumvalue of the size of the local region at which a learning effect relatedto the first hierarchical layer is expected, and being less than aminimum value of the size of the local region at which a learning effectin a second hierarchical layer of the hierarchical layers being locatedat a stage next to the first hierarchical layer is expected.